Muhammad Haris Rao
wDefinition: Let $\mu : \mathcal{B} \left( \mathbb{R} \right) \longrightarrow [0, 1]$ be a probability measure on $\mathbb{R}$. The characteristic function $\phi_\mu : \mathbb{R} \longrightarrow \mathbb{C}$ is defined by $$ \phi_\mu (t) = \int_\mathbb{R} e^{itx} \, d \mu(x) $$ If $X$ is a real valued random variable on a probability space with distribution function $F_X$, then by characteristic function $\phi_X : \mathbb{R} \longrightarrow \mathbb{C}$ of $X$ we mean the characteristic function of the distrubution on $\mathbb{R}$ induced by $X$. That is $$ \phi_X(t) = \mathbf{E} \left[ e^{itX} \right] = \int_\mathbb{R} e^{itx} \, d F_X(x) $$
Lemma: For $\alpha \in \mathbb{R}$, we have $$ \lim_{T \to \infty} \int_0^T \frac{\sin{(\alpha t)}}{t} \, dt = \text{sgn} \left( \alpha \right) \frac{\pi}{2} $$ Furthermore, for all $\alpha \in \mathbb{R}$ and $T \ge 0$, $$ \left| \int_0^T \frac{\sin{(\alpha t)}}{t} \, dt \right| \le \pi $$
Theorem: If $\mu : \mathcal{B} \left( \mathbb{R} \right) \longrightarrow [0, 1]$ is a probability distribution on $\mathbb{R}$, and $a, b \in \mathbb{R}$ are real numbers with $a < b$, then $$ \mu \left( (a, b) \right) + \frac{1}{2} \mu \left( \{ a , b \} \right) = \frac{1}{2 \pi} \lim_{T \to \infty} \int_{-T}^T \frac{e^{-ita} - e^{-itb}}{it} \phi_\mu(t) \, dt $$
Proof. Using the definition $$ \phi_\mu (t) = \int_\mathbb{R} e^{itx} \, d \mu(x) $$ we obtain \begin{align*} \int_{-T}^T \frac{e^{ita} - e^{itb}}{it} \phi_\mu(t) \, dt &= \int_{-T}^T \frac{e^{-ita} - e^{-itb}}{it} \int_\mathbb{R} e^{itx} \, d \mu(x) \, dt \\ &= \int_{-T}^T \int_\mathbb{R} \frac{e^{-ita} - e^{-itb}}{it} e^{itx} \, d \mu(x) \, dt \\ &= \int_{-T}^T \int_\mathbb{R} \frac{e^{it(x - a)} - e^{it(x - b)}}{it} \, d \mu(x) \, dt \end{align*} We have the uniform bound $$ \left| \frac{e^{it(x - a)} - e^{it(x - b)}}{it} \right| \le \left| \frac{it(x - a) - it(x - b)}{it}\right| = \left| \frac{(b - a)t}{t} \right| = b - a $$ so the integrand is bounded on the region of integration which is $[-T, T] \times \mathbb{R}$ with $[-T, T]$ endowed with the usual Lebesgue measure, and $\mathbb{R}$ given the probability measure $\mu$. The product measure is therefore a finite measure, and so the integrand is integrable since it is bounded. So we can apply Fubini's theorem to obtain \begin{align*} \int_{-T}^T \frac{e^{ita} - e^{itb}}{it} \phi_\mu(t) \, dt = \int_{-T}^T \int_\mathbb{R} \frac{e^{it(x - a)} - e^{it(x - b)}}{it} \, d \mu(x) \, dt = \int_\mathbb{R} \int_{-T}^T \frac{e^{it(x - a)} - e^{it(x - b)}}{it} \, dt \, d \mu(x) \end{align*} Expanding the complex exponential into sines and cosines and applying a straightfoward oddness/evenness argument argument for integrals over a symmetric region of integration yields \begin{align*} \int_\mathbb{R} \int_{-T}^T \frac{e^{it(x - a)} - e^{it(x - b)}}{it} \, dt \, d \mu(x) &= \int_\mathbb{R} \int_{-T}^T \frac{\sin{\left(t(x - a)\right)} - \sin{\left(t(x - b)\right)}}{t} \, dt \, d \mu(x) \\ &= 2 \int_\mathbb{R} \int_0^T \frac{\sin{\left(t(x - a) \right)}}{t} \, dt - \int_0^T \frac{\sin{\left(t(x - b) \right)}}{t} \, dt \, d \mu(x) \end{align*} Now we would like to take $T \to \infty$ an interchange the integral and limit in $$ \frac{1}{2 \pi} \lim_{T \to \infty} \int_{-T}^T \frac{e^{ita} - e^{itb}}{it} \phi_\mu(t) \, dt = \frac{1}{\pi} \lim_{T \to \infty} \int_\mathbb{R} \int_0^T \frac{\sin{\left(t(x - a) \right)}}{t} \, dt - \int_0^T \frac{\sin{\left(t(x - b) \right)}}{t} \, dt \, d \mu(x) $$ From our lemma we know that independently of $\alpha \in \mathbb{R}$ and $T \ge 0$, $$ \left| \int_0^T \frac{\sin{(\alpha t)}}{t} \, dt \right| \le \pi $$ Therefore, we have the uniform bound $$ \left| \int_0^T \frac{\sin{\left(t(x - a) \right)}}{t} \, dt - \int_0^T \frac{\sin{\left(t(x - b) \right)}}{t} \, dt \right| \le 2 \pi $$ which is integrable on $\mathbb{R}$ with respect to the finite measure $\mu$. Therefore, we may apply the Lebesgue dominated convergence theorem to obtain $$ \frac{1}{2 \pi} \lim_{T \to \infty} \int_{-T}^T \frac{e^{ita} - e^{itb}}{it} \phi_\mu(t) \, dt = \frac{1}{\pi} \int_\mathbb{R} \int_0^\infty \frac{\sin{\left(t(x - a) \right)}}{t} \, dt \, d \mu(x) - \frac{1}{\pi} \int_\mathbb{R} \int_0^\infty \frac{\sin{\left(t(x - b) \right)}}{t} \, dt \, d \mu(x) \\ $$ The first iterated integral is \begin{align*} \frac{1}{\pi} \int_\mathbb{R} \int_0^\infty \frac{\sin{\left(t(x - a) \right)}}{t} \, dt \, d \mu(x) &= \frac{1}{\pi} \int_{(-\infty, a)} \int_0^\infty \frac{\sin{\left(t(x - a) \right)}}{t} \, dt \, d \mu(x) + \frac{1}{\pi} \int_{\{ a \}} \int_0^\infty \frac{\sin{\left(t(x - a) \right)}}{t} \, dt \, d \mu(x) \\ &\,\,\,\,\,\,\,\,\,\,\,\, + \frac{1}{\pi} \int_{(a, \infty)} \int_0^\infty \frac{\sin{\left(t(x - a) \right)}}{t} \, dt \, d \mu(x) \\ &= - \frac{1}{2} \mu \left( (-\infty, a) \right) + \frac{1}{2} \mu \left( (a, \infty) \right) \end{align*} and the second is \begin{align*} \frac{1}{\pi} \int_\mathbb{R} \int_0^\infty \frac{\sin{\left(t(x - b) \right)}}{t} \, dt \, d \mu(x) &= \frac{1}{\pi} \int_{(-\infty, b)} \int_0^\infty \frac{\sin{\left(t(x - b) \right)}}{t} \, dt \, d \mu(x) + \frac{1}{\pi} \int_{\{ b \}} \int_0^\infty \frac{\sin{\left(t(x - b) \right)}}{t} \, dt \, d \mu(x) \\ &\,\,\,\,\,\,\,\,\,\,\,\, + \frac{1}{\pi} \int_{(b, \infty)} \int_0^\infty \frac{\sin{\left(t(x - b) \right)}}{t} \, dt \, d \mu(x) \\ &= - \frac{1}{2} \mu \left( (-\infty, b) \right) + \frac{1}{2} \mu \left( (b, \infty) \right) \end{align*} So we have finally \begin{align*} \frac{1}{2 \pi} \lim_{T \to \infty} \int_{-T}^T \frac{e^{ita} - e^{itb}}{it} \phi_\mu(t) \, dt &= - \frac{1}{2} \mu \left( (-\infty, a) \right) + \frac{1}{2} \mu \left( (a, \infty) \right) + \frac{1}{2} \mu \left( (-\infty, b) \right) - \frac{1}{2} \mu \left( (b, \infty) \right) \\ &= \frac{1}{2} \left( \mu \left( (-\infty, b) \right) - \mu \left( (-\infty, a) \right) \right) + \frac{1}{2} \left( \mu \left( (a, \infty) \right) - \mu \left( (b, \infty) \right) \right) \\ &= \frac{1}{2} \mu \left( [a, b) \right) + \frac{1}{2} \mu \left( (a, b] \right) \\ &= \mu \left( (a, b) \right) + \frac{1}{2} \mu \left( \{ a, b \} \right) \end{align*} which is what we wanted to show.$\blacksquare$
Corollary: Given two probability measures on $\mu, \nu : \mathcal{B} \left( \mathbb{R} \right) \longrightarrow [0, 1]$, we have $\mu = \nu$ if and only if $\phi_\mu = \phi_\nu$.
Proof. The forward direction is obvious. We will prove the reverse. Define the sets \begin{align*} C_\mu^n = \left\{ x \in \mathbb{R} \mid \mu\left( \{ x \} \right) > \frac{1}{n} \right\} \\ C_\nu^n = \left\{ x \in \mathbb{R} \mid \nu\left( \{ x \} \right) > \frac{1}{n} \right\} \end{align*} for positive integers $n \ge 1$. It is easily seen that $C_\mu^n, C_\nu^n$ are each finite (they have size strictly less than $n$), and so \begin{align*} C_\mu &= \left\{ x \in \mathbb{R} \mid \mu\left( \{ x \} \right) > 0 \right\} = \bigcup_{n \ge 1} C_\mu^n \\ C_\nu &= \left\{ x \in \mathbb{R} \mid \nu\left( \{ x \} \right) > 0 \right\} = \bigcup_{n \ge 1} C_\nu^n \end{align*} are both at most countable. Set $C = \mathbb{R} - (C_\mu \cup C_\nu)$ to be the set of real numbers $x$ such that $\mu(\{ x \}) = 0$. Its complement is $C^c = C_\mu \cup C_\nu$ which is at most countable. Because of this, the open intervals $$ \{ (a, b) \in \mathcal{B} \left( \mathbb{R} \right) \mid a, b \in C \text{ and } a < b \} $$ generate all of $\mathcal{B} \left( \mathbb{R} \right)$. Furthermore, $\mu, \nu$ agree on these open intervals because if $a, b \in C$ with $a < b$ then \begin{align*} \mu \left( (a, b) \right) &= \mu \left( (a, b) \right) + \frac{1}{2} \mu \left( \{ a, b \} \right) \\ &= \frac{1}{2 \pi} \lim_{T \to \infty} \int_{-T}^T \frac{e^{-ita} - e^{-itb}}{it} \phi_\mu(t) \, dt \\ &= \frac{1}{2 \pi} \lim_{T \to \infty} \int_{-T}^T \frac{e^{-ita} - e^{-itb}}{it} \phi_\nu(t) \, dt \\ &= \nu \left( (a, b) \right) + \frac{1}{2} \nu \left( \{ a, b \} \right) \\ &= \nu \left( (a, b) \right) \end{align*} Finally, this generating set is a $\pi$-system, so it follows as a corollary of the $\pi$-$\lambda$ theorem that $\mu, \nu$ agree on the whole $\sigma$-algebra of Borel subsets of $\mathbb{R}$.$\blacksquare$
Thus, we have a one-to-one correspondence between probability distributions and characteristic functions. In the langauge of random variables we are saying that
Corollary: If $X, Y$ are real valued random variables then $X \overset{d}{=} Y$ if and only if $\phi_X = \phi_Y$.
There is also the following result for integrable characteristic functions:
Theorem: Let $\mu : \mathcal{B}(\mathbb{R}) \longrightarrow [0, 1]$ be a probability distribution on $\mathbb{R}$ such that its characteristic function $\phi_\mu : \mathbb{R} \longrightarrow \mathbb{C}$ is absolutely integrable. Then $\mu$ is absolutely continuous with density function $f_\mu : \mathbb{R} \longrightarrow \mathbb{R}$ $$ f_\mu \left( x \right) = \frac{1}{2 \pi} \int_\mathbb{R} e^{-itx} \, \phi_\mu (t) \, dt $$
Proof. From the previous inversion formula, we have that for $a, b \in \mathbb{R}$ such that $a < b$, \begin{align*} \mu((a, b)) + \frac{1}{2} \mu \left( \{ a, b \} \right) &= \frac{1}{2 \pi} \lim_{T \to \infty} \int_{-T}^T \frac{e^{-ita} - e^{-itb}}{it} \phi_\mu(t) \, dt \end{align*} Recalling that $$ \left| \frac{e^{-ita} - e^{-itb}}{it} \right| \le b - a $$ the integrand is integrable on $\mathbb{R}$ so we can get rid of the limit and obtain \begin{align*} \mu((a, b)) + \frac{1}{2} \mu \left( \{ a, b \} \right) &= \frac{1}{2 \pi} \int_\mathbb{R} \frac{e^{-ita} - e^{-itb}}{it} \phi_\mu(t) \, dt \\ &= \frac{1}{2 \pi} \int_\mathbb{R} \left( \int_a^b e^{-itx} \, dx \right) \phi_\mu(t) \, dt \\ &= \frac{1}{2 \pi} \int_\mathbb{R} \int_a^b e^{-itx} \phi_\mu(t) \, dx \, dt \end{align*} Since $\phi_\mu$ is integrable on $\mathbb{R}$, and $|e^{-itx} \phi_\mu(t)| = |\phi_\mu(t)|$, the integrand above is integrable on $\mathbb{R} \times [a, b]$. So by Fubini's theorem, $$ \mu((a, b)) + \frac{1}{2} \mu \left( \{ a, b \} \right) = \int_a^b \left( \frac{1}{2 \pi} \int_\mathbb{R} e^{-itx} \phi_\mu(t) \, dt \right) \, dx $$ Like before, let $C = \{ x \in \mathbb{R} \mid \mu(\{ x \}) = 0 \} \subseteq \mathbb{R}$. Then it can be shown that $\mathbb{R} \backslash C$ is at most countably large, so consequently, the $\pi$-system $\{ (a, b) \mid a, b \in C \text{ and } a < b \}$ will generate all of $\mathcal{B}(\mathbb{R})$. Also, if $a, b \in C$ with $a < b$ then due to the fact that $\mu (\{ a, b \}) = 0$ we will have $$ \mu \left( (a, b) \right) = \int_{(a, b)} \left( \frac{1}{2 \pi} \int_\mathbb{R} e^{-itx} \phi_\mu(t) \, dt \right) \, dx $$ By the $\pi$-$\lambda$ theorem, it follows that for every $B \in \mathcal{B}(\mathbb{R})$, $$ \mu(B) = \int_B \left( \frac{1}{2 \pi} \int_\mathbb{R} e^{-itx} \phi_\mu(t) \, dt \right) \, dx $$ so $\mu$ has density function $$ f_\mu(x) = \frac{1}{2 \pi} \int_\mathbb{R} e^{-itx} \phi_\mu(t) \, dt $$ which is what we wanted to show.$\blacksquare$